Machine Reading Comprehension Research Articles

Speculation and negation identification via unified Machine Reading Comprehension frameworks with lexical and syntactic data augmentation

Speculation and Negation Identification focuses on the extraction of speculative and negative cues and scopes. Previous work relied on complete syntactic trees or simply fed sentences into pre-trained language models, which were confronted with poor generalization within and across datasets, and the limitations of training samples. Accordingly, we build a complete pipeline framework that firstly detects cues and then extracts scopes, and propose unified Machine Reading Comprehension paradigms for both cue detection and scope resolution on several datasets. To tackle the insufficiency of training sets and produce more useful samples with appropriate amount of lexical and syntactic knowledge, we apply data augmentation integrating lexical and syntactic features for scope resolution. Experimental results show that our model achieves higher performance than baselines on several publicly accessible corpora.

Generative large language models are all-purpose text analytics engines: text-to-text learning is all your need.

To solve major clinical natural language processing (NLP) tasks using a unified text-to-text learning architecture based on a generative large language model (LLM) via prompt tuning. We formulated 7 key clinical NLP tasks as text-to-text learning and solved them using one unified generative clinical LLM, GatorTronGPT, developed using GPT-3 architecture and trained with up to 20 billion parameters. We adopted soft prompts (ie, trainable vectors) with frozen LLM, where the LLM parameters were not updated (ie, frozen) and only the vectors of soft prompts were updated, known as prompt tuning. We added additional soft prompts as a prefix to the input layer, which were optimized during the prompt tuning. We evaluated the proposed method using 7 clinical NLP tasks and compared them with previous task-specific solutions based on Transformer models. The proposed approach achieved state-of-the-art performance for 5 out of 7 major clinical NLP tasks using one unified generative LLM. Our approach outperformed previous task-specific transformer models by ∼3% for concept extraction and 7% for relation extraction applied to social determinants of health, 3.4% for clinical concept normalization, 3.4%-10% for clinical abbreviation disambiguation, and 5.5%-9% for natural language inference. Our approach also outperformed a previously developed prompt-based machine reading comprehension (MRC) model, GatorTron-MRC, for clinical concept and relation extraction. The proposed approach can deliver the "one model for all" promise from training to deployment using a unified generative LLM.

CGKPN: Cross-Graph Knowledge Propagation Network with Adaptive Connection for Reasoning-Based Machine Reading Comprehension

CGKPN: Cross-Graph Knowledge Propagation Network with Adaptive Connection for Reasoning-Based Machine Reading Comprehension

Medical knowledge graph question answering for drug‐drug interaction prediction based on multi‐hop machine reading comprehension

AbstractDrug‐drug interaction (DDI) prediction is a crucial issue in molecular biology. Traditional methods of observing drug‐drug interactions through medical experiments require significant resources and labour. The authors present a Medical Knowledge Graph Question Answering (MedKGQA) model, dubbed MedKGQA, that predicts DDI by employing machine reading comprehension (MRC) from closed‐domain literature and constructing a knowledge graph of “drug‐protein” triplets from open‐domain documents. The model vectorises the drug‐protein target attributes in the graph using entity embeddings and establishes directed connections between drug and protein entities based on the metabolic interaction pathways of protein targets in the human body. This aligns multiple external knowledge and applies it to learn the graph neural network. Without bells and whistles, the proposed model achieved a 4.5% improvement in terms of DDI prediction accuracy compared to previous state‐of‐the‐art models on the QAngaroo MedHop dataset. Experimental results demonstrate the efficiency and effectiveness of the model and verify the feasibility of integrating external knowledge in MRC tasks.

SSR: Solving Named Entity Recognition Problems via a Single-stream Reasoner

Information Extraction (IE) focuses on transforming unstructured data into structured knowledge, of which Named Entity Recognition (NER) is a fundamental component. In the realm of Information Retrieval (IR), effectively recognizing entities can substantially enhance the precision of search and recommendation systems. Existing methods frame NER as a sequence labeling task, which requires extra data and, therefore may be limited in terms of sustainability. One promising solution is to employ a Machine Reading Comprehension (MRC) approach for NER tasks, thereby eliminating the dependence on additional data. This process encounters key challenges, including: 1) Unconventional predictions; 2) Inefficient multi-stream processing; 3) Absence of a proficient reasoning strategy. To this end, we present the Single-Stream Reasoner (SSR), a solution utilizing a reasoning strategy and standardized inputs. This yields a type-agnostic solution for both flat and nested NER tasks, without the need for additional data. On ten NER benchmarks, SSR achieved state-of-the-art results, highlighting its robustness. Furthermore, we illustrated its efficiency through convergence, inference speed, and low-resource scenario performance comparisons. Our architecture displays adaptability and can effortlessly merge with various foundational models and reasoning strategies, fostering advancements in both IR and IE fields.

面向机器阅读理解的医学域数据集MedicalQA

Machine reading comprehension aims to make the computer understand the paragraph semantics and answer the questions raised by users using algorithms. The quality of the dataset used in this task can directly affect the experimental results of the model. In order to enrich the medical domain dataset of machine reading comprehension, this paper constructs MedicalQA, a medical domain dataset for machine reading comprehension, employing a combination of web crawlers and manual annotation techniques. The dataset takes two medical platforms (i.e. Xunyiwenyao Network and 39 Health Network) as main data sources, and includes 19,502 paragraphs and Q & A pairs, covering 9 medical departments, such as internal medicine, surgery, obstetrics and gynecology. The dataset is formatted as an Excel file, organized with 5e columns. The first column denotes the paragraph ID; the second column indicates the department to which the paragraph belongs; the third column contains the paragraph content; the fourth column lists the questions, and the fifth column provides corresponding answers to the questions. The construction of this dataset is conducive to the establishment of machine reading comprehension models in the medical domain, and can also promote the sharing of medical datasets in the field of machine reading comprehension.

Product information recognition in the retail domain as an MRC problem

This paper presents the task of recognizing product information (PI) (i.e., product names, prices, materials, etc.) mentioned in customer statements. This is one of the key components in developing artificial intelligence products to enable businesses to listen to their customers, adapt to market dynamics, continuously improve their products and services, and improve customer engagement by enhancing effectiveness of a chatbot. To this end, natural language processing (NLP) tools are commonly used to formulate the task as a traditional sequence labeling problem. However, in this paper, we bring the power of machine reading comprehension (MRC) tasks to propose another, alternative approach. In this setting, determining product information types is the same as asking “Which PI types are referenced in the statement?” For example, extracting product names (which corresponds to the label PRO_NAME) is cast as retrieving answer spans to the question “Which instances of product names are mentioned here?” We perform extensive experiments on a Vietnamese public dataset. The experimental results show the robustness of the proposed alternative method. It boosts the performance of the recognition model over the two robust baselines, giving a significant improvement. We achieved 92.87% in the F1 score on recognizing product descriptions at Level 1. At Level 2, the model yielded 93.34% in the F1 score on recognizing each product information type.

Efficient Machine Reading Comprehension for Health Care Applications: Algorithm Development and Validation of a Context Extraction Approach.

Extractive methods for machine reading comprehension (MRC) tasks have achieved comparable or better accuracy than human performance on benchmark data sets. However, such models are not as successful when adapted to complex domains such as health care. One of the main reasons is that the context that the MRC model needs to process when operating in a complex domain can be much larger compared with an average open-domain context. This causes the MRC model to make less accurate and slower predictions. A potential solution to this problem is to reduce the input context of the MRC model by extracting only the necessary parts from the original context. This study aims to develop a method for extracting useful contexts from long articles as an additional component to the question answering task, enabling the MRC model to work more efficiently and accurately. Existing approaches to context extraction in MRC are based on sentence selection strategies, in which the models are trained to find the sentences containing the answer. We found that using only the sentences containing the answer was insufficient for the MRC model to predict correctly. We conducted a series of empirical studies and observed a strong relationship between the usefulness of the context and the confidence score output of the MRC model. Our investigation showed that a precise input context can boost the prediction correctness of the MRC and greatly reduce inference time. We proposed a method to estimate the utility of each sentence in a context in answering the question and then extract a new, shorter context according to these estimations. We generated a data set to train 2 models for estimating sentence utility, based on which we selected more precise contexts that improved the MRC model's performance. We demonstrated our approach on the Question Answering Data Set for COVID-19 and Biomedical Semantic Indexing and Question Answering data sets and showed that the approach benefits the downstream MRC model. First, the method substantially reduced the inference time of the entire question answering system by 6 to 7 times. Second, our approach helped the MRC model predict the answer more correctly compared with using the original context (F1-score increased from 0.724 to 0.744 for the Question Answering Data Set for COVID-19 and from 0.651 to 0.704 for the Biomedical Semantic Indexing and Question Answering). We also found a potential problem where extractive transformer MRC models predict poorly despite being given a more precise context in some cases. The proposed context extraction method allows the MRC model to achieve improved prediction correctness and a significantly reduced MRC inference time. This approach works technically with any MRC model and has potential in tasks involving processing long texts.

PMRC: Prompt-Based Machine Reading Comprehension for Few-Shot Named Entity Recognition

The prompt-based method has been proven effective in improving the performance of pre-trained language models (PLMs) on sentence-level few-shot tasks. However, when applying prompting to token-level tasks such as Named Entity Recognition (NER), specific templates need to be designed, and all possible segments of the input text need to be enumerated. These methods have high computational complexity in both training and inference processes, making them difficult to apply in real-world scenarios. To address these issues, we redefine the NER task as a Machine Reading Comprehension (MRC) task and incorporate prompting into the MRC framework. Specifically, we sequentially insert boundary markers for various entity types into the templates and use these markers as anchors during the inference process to differentiate entity types. In contrast to the traditional multi-turn question-answering extraction in the MRC framework, our method can extract all spans of entity types in one round. Furthermore, we propose word-based template and example-based template that enhance the MRC framework's perception of entity start and end positions while significantly reducing the manual effort required for template design. It is worth noting that in cross-domain scenarios, PMRC does not require redesigning the model architecture and can continue training by simply replacing the templates to recognize entity types in the target domain. Experimental results demonstrate that our approach outperforms state-of-the-art models in low-resource settings, achieving an average performance improvement of +5.2% in settings where access to source domain data is limited. Particularly, on the ATIS dataset with a large number of entity types and 10-shot setting, PMRC achieves a performance improvement of +15.7%. Moreover, our method achieves a decoding speed 40.56 times faster than the template-based cloze-style approach.

Mitigating Idiom Inconsistency: A Multi-Semantic Contrastive Learning Method for Chinese Idiom Reading Comprehension

Chinese idioms pose a significant challenge for machine reading comprehension due to their metaphorical meanings often diverging from their literal counterparts, leading to metaphorical inconsistency. Furthermore, the same idiom can have different meanings in different contexts, resulting in contextual inconsistency. Although deep learning-based methods have achieved some success in idioms reading comprehension, existing approaches still struggle to accurately capture idiom representations due to metaphorical inconsistency and contextual inconsistency of idioms. To address these challenges, we propose a novel model, Multi-Semantic Contrastive Learning Method (MSCLM), which simultaneously addresses metaphorical inconsistency and contextual inconsistency of idioms. To mitigate metaphorical inconsistency, we propose a metaphor contrastive learning module based on the prompt method, bridging the semantic gap between literal and metaphorical meanings of idioms. To mitigate contextual inconsistency, we propose a multi-semantic cross-attention module to explore semantic features between different metaphors of the same idiom in various contexts. Our model has been compared with multiple current latest models (including GPT-3.5) on multiple Chinese idiom reading comprehension datasets, and the experimental results demonstrate that MSCLM outperforms state-of-the-art models.

Benchmarks for Pirá 2.0, a Reading Comprehension Dataset about the Ocean, the Brazilian Coast, and Climate Change

ABSTRACT Pirá is a reading comprehension dataset focused on the ocean, the Brazilian coast, and climate change, built from a collection of scientific abstracts and reports on these topics. This dataset represents a versatile language resource, particularly useful for testing the ability of current machine learning models to acquire expert scientific knowledge. Despite its potential, a detailed set of baselines has not yet been developed for Pirá. By creating these baselines, researchers can more easily utilize Pirá as a resource for testing machine learning models across a wide range of question answering tasks. In this paper, we define six benchmarks over the Pirá dataset, covering closed generative question answering, machine reading comprehension, information retrieval, open question answering, answer triggering, and multiple choice question answering. As part of this effort, we have also produced a curated version of the original dataset, where we fixed a number of grammar issues, repetitions, and other shortcomings. Furthermore, the dataset has been extended in several new directions, so as to face the aforementioned benchmarks: translation of supporting texts from English into Portuguese, classification labels for answerability, automatic paraphrases of questions and answers, and multiple choice candidates. The results described in this paper provide several points of reference for researchers interested in exploring the challenges provided by the Pirá dataset.

Combining permuted language model and adversarial training for Chinese machine reading comprehension

Due to the polysemy and complexity of the Chinese language, Chinese machine reading comprehension has always been a challenging task. To improve the semantic understanding and robustness of Chinese machine reading comprehension models, we propose a model that utilizes adversarial training algorithms and Permuted Language Model (PERT). Firstly, we employ the PERT pre-training model to embed paragraphs and questions into vector space to obtain corresponding sequential representations. Secondly, we use a multi-head self-attention mechanism to extract key textual information from the sequence and employ a Bi-GRU network to semantically fuse the output feature vectors, aiming to learn deep semantic representations in the text. Finally, we introduce perturbations into the model training process. We achieve this by utilizing adversarial training algorithms such as Fast Gradient Method (FGM) and Projected Gradient Descent (PGD). These algorithms generate adversarial samples to enhance the model’s robustness and stability when facing diverse inputs. We conducted comparative experiments on the publicly available Chinese reading comprehension datasets CMRC2018 and DRCD. The experimental results show that our proposed model has achieved significant improvements in both EM and F1-Score compared to the baseline model. To validate the model’s generalization and robustness, we utilized ChatGPT to construct a scientific dataset that includes a large number of domain-specific terms, sentences with mixed Chinese and English, and complex comprehension tasks. Our model also performed remarkably well on the self-built dataset. In conclusion, the proposed model not only effectively enhances the understanding of semantic information in Chinese text but also demonstrates a certain level of generalization capability.

A machine reading comprehension framework for recognizing emotion cause in conversations

Recognizing Emotion Cause in Conversations (RECC) is a key issue in modeling human cognitive processes, involving Conversational Causal Emotion Entailment task (C2E2) and Conversational Causal Span Extraction task (C2SE). Previous emotion cause extraction research has been concentrated at the clause level, detecting if the cause is in the text, not describing the underlying causes in texts well. In order to address this issue, we suggest a novel approach that can recognize emotion cause spans. These spans can represent or imply the causes for controlling emotions. In this paper, we use a Machine Reading Comprehension framework to Recognize the Emotion Cause in Conversations (MRC-RECC), at both the span level and clause level simultaneously. Specifically, we use two types of queries to build the associations between the two different subtasks: emotion causal entailment task and emotion causal span extraction task. Our framework can recognize emotion cause more effectively by using joint learning to make these two tasks complement each other. Experiments demonstrate that our MRC-RECC provides state-of-the-art performances, which can reason more emotion causes in conversation texts. The code can be found at https://github.com/Guangzidetiaoyue/MRC-RECCON.

Genre: generative multi-turn question answering with contrastive learning for entity–relation extraction

AbstractExtractive approaches have been the mainstream paradigm for identifying overlapping entity–relation extraction. However, limited by their inherently methodological flaws, which hardly deal with three issues: hierarchical dependent entity–relations, implicit entity–relations, and entity normalization. Recent advances have proposed an effective solution based on generative language models, which cast entity–relation extraction as a sequence-to-sequence text generation task. Inspired by the observation that humans learn by getting to the bottom of things, we propose a novel framework, namely GenRE, Generative multi-turn question answering with contrastive learning for entity–relation extraction. Specifically, a template-based question prompt generation first is designed to answer in different turns. We then formulate entity–relation extraction as a generative question answering task based on the general language model instead of span-based machine reading comprehension. Meanwhile, the contrastive learning strategy in fine-tuning is introduced to add negative samples to mitigate the exposure bias inherent in generative models. Our extensive experiments demonstrate that GenRE performs competitively on two public datasets and a custom dataset, highlighting its superiority in entity normalization and implicit entity–relation extraction. (The code is available at https://github.com/lovelyllwang/GenRE).

A machine reading comprehension model with counterfactual contrastive learning for emotion-cause pair extraction

A machine reading comprehension model with counterfactual contrastive learning for emotion-cause pair extraction

Effective history and efficient integration for conversational machine reading comprehension

 The task of conversational machine reading comprehension (CMRC) is an extension of single-turn machine reading comprehension to multi-turn settings, to relflect the conversational way in which people seek information. The correlations between multiple rounds of questions mean that the conversation history is critical to solving the CMRC task. However, existing CMRC models ignore the interference that arises from using excessive historical information to answer the current question when incorporating the dialogue history into the current question. In this paper, an effective Question Selection Module (QSM) is designed to select most relevant historical dialogues when answering the current question through question coupling and coarse-to-fine matching. In addition, most existing approaches perform memory inference by stacked RNNs at context word level, without considering semantic information flowing in the direction of conversation flow. In view of this problem, we implement sequential recurrent reasoning at the turn level of the dialogue, where the turn information contains all the filtered historical semantics for the current step. We conduct experiments on two benchmark datasets, QuAC and CoQA, released by Stanford University. The results confirm that our model satisfactorily captures the valid history and performs recurrent reasoning, and our model achieves an F1-score of 83.0% on CoQA dataset and 67.8% on QuAC dataset, outperforming the best alternative model by 4.6% on CoQA and 2.7% on QuAC.

Chinese EMR Named Entity Recognition Using Fused Label Relations Based on Machine Reading Comprehension Framework.

Chinese electronic medical records (EMR) presents significant challenges for named entity recognition (NER) due to their specialized nature, unique language features, and diverse expressions. Traditionally, NER is treated as a sequence labeling task, where each token is assigned a label. Recent research has reframed NER within the machine reading comprehension (MRC) framework, extracting entities in a question-answer format, achieving state-of-the-art performance. However, these MRC-based methods have a significant limitation: they extract entities of various types independently, ignoring their interrelations. To address this, we introduce the Fusion Label Relations with MRC (FLR-MRC) model, which enhances the MRC model by implicitly capturing dependencies among entity types. FLR-MRC models interrelations between labels using graph attention networks, integrating these with textual data to identify entities. On the benchmark CMeEE and CCKS2017-CNER datasets, FLR-MRC achieves F1-scores of 0.6652 and 0.9101, respectively, outperforming existing clinical NER methods.

Multi-Task Learning of Japanese How-to Tip Machine Reading Comprehension by a Generative Model

Multi-Task Learning of Japanese How-to Tip Machine Reading Comprehension by a Generative Model

Visual Explanation for Open-domain Question Answering with BERT.

Open-domain question answering (OpenQA) is an essential but challenging task in natural language processing that aims to answer questions in natural language formats on the basis of large-scale unstructured passages. Recent research has taken the performance of benchmark datasets to new heights, especially when these datasets are combined with techniques for machine reading comprehension based on Transformer models. However, as identified through our ongoing collaboration with domain experts and our review of literature, three key challenges limit their further improvement: (i) complex data with multiple long texts, (ii) complex model architecture with multiple modules, and (iii) semantically complex decision process. In this paper, we present VEQA, a visual analytics system that helps experts understand the decision reasons of OpenQA and provides insights into model improvement. The system summarizes the data flow within and between modules in the OpenQA model as the decision process takes place at the summary, instance and candidate levels. Specifically, it guides users through a summary visualization of dataset and module response to explore individual instances with a ranking visualization that incorporates context. Furthermore, VEQA supports fine-grained exploration of the decision flow within a single module through a comparative tree visualization. We demonstrate the effectiveness of VEQA in promoting interpretability and providing insights into model enhancement through a case study and expert evaluation.

Multi-Passage Machine Reading Comprehension through Multi-Task Learning and Dual Verification

Multi-Passage Machine Reading Comprehension through Multi-Task Learning and Dual Verification